Many storage environments may implement functionality to improve storage efficiency. Deduplication is a technique for storage efficiency and can be in the form of post process deduplication and inline deduplication. For example, a storage controller may store data within a storage device. Post-process deduplication may be performed to remove redundant data within the storage device after the data has already been stored within the storage device. In particular, a deduplication scanner detects and eliminates redundant data by scanning all recent data within the storage device. Post-process deduplication has less of a latency impact on a write path of the storage controller because deduplication is not performed inline within the write path over which the storage controller receives and processes write requests from clients. However, additional storage must be provided within the storage device to initially store the redundant data. Also, undesirable on-disk fragmentation occurs as redundant data is freed from the storage device, which can impact I/O on disk based storage devices. Post-process deduplication requires additional system resources to identify redundant data, such as resources used to perform sorting, maintaining hashing tables, etc. While post-process deduplication is being performed, regular I/O to the storage device can be impacted, thus clients may experience unpredictable performance degradation.
Inline deduplication can be performed in the write path in order to remove redundant data before the redundant data reaches the storage device. Inline deduplication does not require the additional overhead and resources used by post-process deduplication because redundant data is eliminated in the write path before reaching the storage device. Inline deduplication also eliminates redundant data without creating on-disk fragmentation since merely unique data is written to the storage device. However, inline deduplication can impact latency of write requests by clients because deduplication is performed within the write path. Thus, write requests can be completed and responded back to clients by the storage controller in a delayed manner.